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Amendments to the claims: 



1 . (original) A method for masking a foreground portion of a digital image from a 
background portion of a digital image, the digital image being part of a video comprising a time 
sequence of digital images, each image being defined by a plurality of pixels, the method 
comprising: 

receiving a first input defining a first border region, the first border region including at 
least a part of the foreground portion and at least a part of the background portion in a first 
digital image; 

receiving a second input defining a second border region, the second border region 
including at least a part of the foreground portion and at least a part of the background portion in 
a second digital image; 

interpolating an intermediary border region for an image intermediary in time to the first 
and second digital images; and 

using the first, second, and intermediary border regions for masking the foreground 
portion from the background portion in the digital video. 

2. (original) The method of claim 1 wherein receiving a first and a second input 
comprise: 

receiving user inputs defining the border regions; and where interpolating an 
intermediary border region comprises: 

interpolating an intermediary border region automatically without user input. 



3. (original) The method of claim 1 wherein using the first, second, and intermediary 
border regions for masking the foregroimd portion from the background portion comprises: 

determining for a pixel in a border region whether it includes data that is associated with 
the foreground portion; and 

using the result of the determining step to mask the foreground portion from the 
background portion in the digital video. 
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4. (original) The method of claim 1, further comprising the steps of: 

estimating an intrinsic color value for a pixel in the first, second, and intermediary border 
regions; and 

using the estimated intrinsic color value for extracting the foregroimd portion fi-om the 
background portion. 

5. (original) The method of claim 1 wherein receiving inputs indicating the border 
regions comprises: 

generating an inside path located inside a foreground portion; and 

generating an outside path located outside the foreground portion and enclosing the inside 

path, 

wherein pixels between the inside and outside paths belong to a border region. 

6. (original) The method of claim 5 wherein generating an inside path and an outside 
path comprise: 

generating a vector-based inside path and a vector-based outside path. 

7. (original) The method of claim 2 wherein receiving inputs indicating the border 
regions comprises: 

generating a single path having a width encompassing that of the border region. 

8. (original) The method of claim 7 wherein generating a single path comprises: 
generating a vector-based single path. 

9. (original) The method of claim 7 wherein generating a single path comprises: 
generating a single path having variable thickness throughout its length. 



10. (original) The method of claim 1 wherein using the first, second, and intermediary 
border regions for masking the foreground portion from the background portion comprises: 
masking the foreground portion and a portion of the border region; and 
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applying Gaussian blur to the portion of the border region. 

11. (original) The method of claim 10, further comprising displaying the masked 
foreground portion and the portion of the border region. 

12. (original) The method of claim 1 wherein using the first, second, and intermediary 
border regions for masking the foreground portion from the background portion comprises: 

masking the foreground portion and a portion of the border region; 
determining the opacities of the pixels in the portion of the border region that is displayed 
with the foreground portion; and 

changing the opacities for pixels having opacities greater than a threshold value. 

13. (original) The method of claim 12, further comprising displaying the masked 
foreground portion and the portion of the border region. 

14. (original) The method of claim 1 further comprising: 

receiving an input defining a border region for a second foreground portion in the first 
digital image, the border region including at least a part of a second foregroxmd portion and at 
least a part of the background portion; 

receiving an input defining a border region for the second foreground portion in the 
second digital image, the border region including at least a part of the second foreground portion 
and at least a part of the background portion; and 

interpolating an intermediary border region for the second foreground portion for an 
image intermediary to the first and second digital images. 

15. (original) The method of claim 1 further comprising: 

receiving an input defining a first intemal border region, the first internal border region 
being enclosed by the first border region, in a first digital image; 

receiving an input defining a second intemal border region, the second intemal border 
region being enclosed by the second border region, in a second digital image; 
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interpolating an intermediary internal border region for an image intermediary in time to 
the first and second digital images; and 

using the first, second, and intermediary border regions, and the first intemal, second 
internal, and intermediary intemal border regions for masking the foreground portion fi'om the 
background portion in the digital video. 

16. (original) The method of claim 15, wherein the first intemal border region, the 
second intemal border region, and the intermediary intemal border region comprise at least a part 
of the foreground portion. 

17. (original) The method of claim 15, wherein the first intemal border region, the 
second intemal border region, and the intermediary intemal border region comprise at least a part 
of the background portion. 



18. (original) The method of claim 15, wherein the first intemal border region, the 
second intemal border region, and the intermediary intemal border region comprise at least a part 
of the foreground portion and at least a part of the background portion. 

19. (original) The method of claim 1, wherein the foreground portion is an object, 

20. (original) A method for masking a foreground portion of a digital image fi-om a 
background portion of a digital image, the digital image being part of a video comprising a time 
sequence of digital images, each image being defined by a plurality of pixels, the method 
comprising: 

receiving an input defining an initial border region, the initial border region including at 
least a part of the foreground portion and at least a part of the background portion in an initial 
digital image; 

automatically generating a border region for another digital image in the time sequence 
based on the initial border region; and 
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using the initial and automatically generated border regions for masking the foreground 
portion from the background portion in the digital video. 

21. (original) The method of claim 20 wherein receiving an input comprises receiving a 
user input defining the border region. 



22. (original) The method of claim 20 wherein using the initial and automatically 
determined border regions for masking the foreground portion from the background portion 
comprises: 

determining for a pixel in a border region whether it comprises data that is associated 
with the foreground portion; and 

using the result of the determining step to mask the foregroimd portion from the 
background portion in the digital video. 

23. (original) The method of claim 20 ftirther comprising: 

estimating an intrinsic color value for a pixel in the first, second, and intermediary border 
regions; and 

using the estimated intrinsic color value for extracting the foreground portion from the 
background portion. 

24. (original) The method of claim 21 wherein receiving a user input indicating the 
border regions comprises: 

generating an inside path located inside the foreground portion; and 
generating an outside path located outside the foreground portion and enclosing the inside 
path, wherein pixels between the inside and outside paths belong to a border region. 
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25. (original) The method of claim 24 wherein generating an inside path and an outside 
path comprise: 

generating a vector-based inside path and a vector-based outside path. 

26. (original) The method of claim 21 wherein receiving a user input indicating the 
border region comprises; 

generating a single path having a width encompassing the border region. 

27. (original) The method of claim 26 wherein generating a single path comprises: 
generating a vector-based single path. 

28. (original) The method of claim 26 wherein generating a single path comprises: 
generating a single path having a variable thickness throughout its length. 

29. (original) The method of claim 20 wherein using the initial and the automatically 
generated border regions for masking the foreground portion from the backgroimd portion 
comprises: 

masking the foreground portion and a portion of the border region; and 
applying Gaussian blur to the portion of the border region. 

30. (original) The method of claim 29 further comprising displaying the masked 
foreground portion and the portion of the border region. 

31. (original) The method of claim 20 wherein using the initial and the automatically 
generated border regions for masking the foreground portion from the background portion 
comprises: 

masking the foreground portion and a portion of the border region; 
determining the opacities of the pixels in the portion of the border region that is displayed 
with the foreground portion; and 

changing the opacities for pixels having opacities greater than a threshold value. 
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32. (original) The method of claim 31, further comprising displaying the masked 
foreground portion and the portion of the border region. 



33. (original) The method of claim 20 further comprising: 

receiving an input defining an initial border region for a second foreground portion, the 
border region including at least a part of a second foreground portion and at least a part of the 
background portion in the digital image with the first foreground portion; 

automatically generating a border region for the second foreground portion in another 
image in the time sequence based on the initial border region for the second foreground portion; 
and 

using the initial and automatically generated border regions for masking the second 
foreground portion fi-om the background portion in the digital video. 



34. (original) The method of claim 20 further comprising: 

receiving an input defining an initial internal border region, the initial internal border 
region being enclosed by the initial border region, in a first digital image; 

automatically generating an internal border region for another image in the time sequence 
based on the initial internal border region; and 

using the initial internal and the automatically generated intemal border regions for 
masking the foreground portion fi-om the background portion in the digital video. 

35. (original) The method of claim 34, wherein the initial intemal border region and the 
automatically generated intemal border region comprise at least a part of the foregroimd portion. 

36. (original) The method of claim 34, wherein the initial intemal border region and the 
automatically generated intemal border region comprise at least a part of the background portion. 

37. (original) The method of claim 34, wherein the initial intemal border region and the 
automatically generated intemal border region comprise at least a part of the foreground portion 
and at least a part of the background portion. 
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38. (original) The method of claim 20, wherein the foregroimd portion is an object. 

39. (original) The method of claim 20 wherein automatically generating a border region 
for another digital image comprises: 

identifying an edge path in the initial border region, the edge path indicating edges of the 
foreground portion in the initial digital image; 

identifying an edge path in an automatically generated border region, the edge path 
indicating edges of the foreground portion in the another digital image; 

determining an edge path transformation between the initial digital image and the another 
digital image; and 

applying the edge path transformation to the inside path and the outside path in the initial 
digital image in order to generate a border region in the another digital image. 

40. (original) The method of claim 39 wherein identifying the edge path in the another 
automatically generated border region comprises: 

identifying the edge path using a snake algorithm operating on a previously identified 
edge path. 

41. (original) The method of claim 39 wherein identifying an edge path in the initial 
border region comprises: 

finding points in the border region that have image gradient values exceeding a threshold 

value. 

42. (currently amended) The method of claim 41 further comprising th e st e p : 
calculating an image gradient value for each color chaimel in the initial border region. 

43. (original) The method of claim 39 wherein identifying an edge path in the initial 
border region comprises: 

copying the inside path and the outside path; 
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moving the copies of the inside path and the outside path towards each other and towards 
edges of the foreground portion using a snake algorithm; and 

identifying the edge path as the path where copy of the inside path and the copy of the 
outside path converge into one path. 

44. (original) The method of claim 39 wherein determining an edge path transformation 
comprises: 

determining a set of local edge path transformations, the set of local edge path 
transformations together forming the edge path transformation. 

45. (original) The method of claim 44 wherein determining a set of local edge path 
transformations comprises: 

determining a local edge path transformation for each point in the edge path. 

46. (original) The method of claim 44 wherein determining a set of local edge path 
transformations comprises: 

determining a translation, a rotation, and a scaling for each local edge path transformation 
in the set of local edge path transformations. 

47. (original) The method of claim 39 wherein applying the edge path transformation to 
the inside and outside path comprises: 

applying a set of local transformations to each point in the inside path and to each point 
in the outside path, the set of local transformations forming the edge path transformation. 



48. (original) The method of claim 47 wherein applying a set of local transformations 
comprises: 

applying a translation, a rotation, and a scaling as each local transformation of the inside 
path and the outside path. 
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49. (original) A method for masking an object from a background in a digital video, the 
digital video comprising a time sequence of digital images, each image being defined by a 
plurality of pixels, the method comprising: 

receiving a first user input defining a first border region, the first border region including 
at least a part of the object and at least a part of the background in a first digital image; 

receiving a second user input defining a second border region, the second border region 
including at least a part of the object and at least a part of the background portion in a second 
digital image; 

interpolating an intermediary border region for an image intermediary in time to the first 
and second digital images; 

using the first, second, and intermediary border regions for masking the object from the 
background in the digital video; 

using the first, second, and intermediary border regions for extracting the object from the 
background in the digital video; and 

placing the masked and extracted object against a new background in another digital 

video. 

50. (original) Computer software, tangibly embodied in a computer-readable medium or 
a propagated carrier signal for masking a foreground portion of a digital image from a 
background portion of a digital image, the digital image being part of a video comprising a time 
sequence of digital images, each image being defined by a plurality of pixels, the software 
comprising instructions to perform the following operations: 

receive a first input defining a first border region, the first border region including at least 
a part of the foreground portion and at least a part of the background portion in a first digital 
image; 

receive a second input defining a second border region, the second border region 
including at least a part of the foregroimd portion and at least a part of the background portion in 
a second digital image; 

interpolate an intermediary border region for an image interinediary in time to the first 
and second digital images; and 
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use the first, second, and intermediary border regions for masking the foreground portion 
from the background portion in the digital video. 

51 . (original) Computer software, tangibly embodied in a computer-readable medium or 
a propagated carrier signal for masking a foreground portion of a digital image from a 
background portion of a digital image, the digital image being part of a video comprising a time 
sequence of digital images, each image being defined by a plurality of pixels, the software 
comprising instructions to perform the following operations: 



including at least a part of the foreground portion and at least a part of the background portion in 
an initial digital image; 

automatically generate a border region for another digital image in the time 
sequence based on the initial border region; and 

use the initial and automatically generated border regions for masking the 
foreground portion from the background portion in the digital video. 

52. (new) The computer software of claim 50 wherein the instructions to receive a first 
and a second input comprise instructions to: 

receive user inputs defining the border regions; and where interpolating an intermediary 
border region comprises: 

interpolate an intermediary border region automatically without user input. 

53. (new) The computer software of claim 50 wherein the instructions to use the first, 
second, and intermediary border regions for masking the foreground portion from the 
background portion comprises instructions to: 

determine for a pixel in a border region whether it includes data that is associated with 
the foreground portion; and 

use the result of the determining step to mask the foreground portion from the 
background portion in the digital video. 



receive an input defining an initial border region, the initial border region 
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54. (new) The computer software of claim 50, further comprising instructions to perform 
the following operations: 

estimate an intrinsic color value for a pixel in the first, second, and intermediary border 
regions; and 

use the estimated intrinsic color value for extracting the foreground portion from the 
background portion. 



55. (new) The computer software of claim 50 wherein the instructions to receive inputs 
indicating the border regions comprise instructions to: 

generate an inside path located inside a foreground portion; and 

generate an outside path located outside the foreground portion and enclosing the inside 

path, 

wherein pixels between the inside and outside paths belong to a border region. 

56. (new) The computer software of claim 55 wherein the instructions to generate an 
inside path and an outside path comprise instructions to generate a vector-based inside path and a 
vector-based outside path. 

57. (new) The computer software of claim 52 wherein the instructions to receive inputs 
indicating the border regions comprise instructions to generate a single path having a width 
encompassing that of the border region. 

58. (new) The computer software of claim 57 wherein the instructions to generate a 
single path comprise instructions to generate a vector-based single path. 

59. (new) The computer software of claim 57 wherein the instructions to generate a 
single path comprise instructions to generate a single path having variable thickness throughout 
its length. 
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60. (new) The computer software of claim 50 wherein the instructions to use the first, 
second, and intermediary border regions for masking the foreground portion from the 
background portion comprise instructions to: 

mask the foreground portion and a portion of the border region; and 
apply Gaussian blur to the portion of the border region. 

61. (new) The computer software of claim 60, further comprising instructions to display 
the masked foreground portion and the portion of the border region. 

62. (new) The computer software of claim 50 wherein the instructions to use the first, 
second, and intermediary border regions for masking the foreground portion from the 
background portion comprise instructions to: 

mask the foreground portion and a portion of the border region; 

determine the opacities of the pixels in the portion of the border region that is displayed 
with the foreground portion; and 

change the opacities for pixels having opacities greater than a threshold value. 

63. (new) The computer software of claim 62, fixrther comprising instructions to display 
the masked foregroimd portion and the portion of the border region. 

64. (new) The computer software of claim 50 fiirther comprising instructions to perform 
the following operations: 

receive an input defining a border region for a second foreground portion in the first 
digital image, the border region including at least a part of a second foreground portion and at 
least a part of the background portion; 

receive an input defining a border region for the second foreground portion in the second 
digital image, the border region including at least a part of the second foregroimd portion and at 
least a part of the background portion; and 

interpolate an intermediary border region for the second foreground portion for an image 
intermediary to the first and second digital images. 
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65. (new) The computer software of claim 50 fiirther comprising instructions to perform 
the following operations: 

receive an input defining a first internal border region, the first internal border region 
being enclosed by the first border region, in a first digital image; 

receive an input defining a second internal border region, the second intemal border 
region being enclosed by the second border region, in a second digital image; 

interpolate an intermediary intemal border region for an image intermediary in time to 
the first and second digital images; and 

use the first, second, and intermediary border regions, and the first intemal, second 
intemal, and intermediary intemal border regions for masking the foreground portion fi*om the 
background portion in the digital video. 

66. (new) The computer software of claim 65, wherein the first intemal border region, 
the second intemal border region, and the intermediary intemal border region comprise at least a 
part of the foreground portion. 

67. (new) The computer softAvare of claim 65, wherein the first intemal border region, 
the second intemal border region, and the intermediary intemal border region comprise at least a 
part of the background portion. 

68. (new) The computer software of claim 65, wherein the first intemal border region, 
the second intemal border region, and the intermediary intemal border region comprise at least a 
part of the foreground portion and at least a part of the background portion. 

69. (new) The computer software of claim 50, wherein the foreground portion is an 

object. 

70. (new) The computer software of claim 51 wherein the instmctions to receive an input 
comprise instmctions to receive a user input defining the border region. 
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71 . (new) The computer software of claim 5 1 wherein the instructions to use the initial 
and automatically determined border regions for masking the foreground portion jfrom the 
background portion comprise instructions to: 

determine for a pixel in a border region whether it comprises data that is associated with 
the foreground portion; and 

use the resuh of the determining step to mask the foregrovmd portion from the 
background portion in the digital video. 

72. (new) The computer software of claim 51 further comprising instructions to perform 
the following operations: 

estimate an intrinsic color value for a pixel in the first, second, and intermediary border 
regions; and 

use the estimated intrinsic color value for extracting the foreground portion from the 
background portion. 

73. (new) The computer software of claim 70 wherein the instructions to receive a user 
input indicating the border regions comprise instructions to: 

generate an inside path located inside the foreground portion; and 
generate an outside path located outside the foreground portion and enclosing the inside 
path, wherein pixels between the inside and outside paths belong to a border region. 

74. (new) The computer software of claim 73 wherein the instructions to generate an 
inside path and an outside path comprise instructions to generate a vector-based inside path and a 
vector-based outside path. 

75. (new) The computer software of claim 70 wherein the instructions to receive a user 
input indicating the border region comprise instructions to generate a single path having a width 
encompassing the border region. 
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76. (new) The computer software of claim 75 wherein the instructions to generate a 
single path comprise instructions to generate a vector-based single path. 

77. (new) The computer software of claim 75 wherein the instructions to generate a 
single path comprise instructions to generate a single path having a variable thickness throughout 
its length. 

78. (new) The computer software of claim 51 wherein the instructions to use the initial 
and the automatically generated border regions for masking the foreground portion from the 
background portion comprise instructions to: 

mask the foreground portion and a portion of the border region; and 
apply Gaussian blur to the portion of the border region. 

79. (new) The computer software of claim 78 fiirther comprising instructions to display 
the masked foreground portion and the portion of the border region. 

80. (new) The computer software of claim 51 wherein the instructions to use the initial 
and the automatically generated border regions for masking the foreground portion from the 
background portion comprise instructions to; 

mask the foreground portion and a portion of the border region; 

determine the opacities of the pixels in the portion of the border region that is displayed 
with the foreground portion; and 

change the opacities for pixels having opacities greater than a threshold value. 

81 . (new) The computer software of claim 80, further comprising instructions to display 
the masked foreground portion and the portion of the border region. 



82. (new) The computer software of claim 51 fiirther comprising instructions to perform 
the following operations: 
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receive an input defining an initial border region for a second foreground portion, the 
border region including at least a part of a second foreground portion and at least a part of the 
background portion in the digital image with the first foreground portion; 

automatically generate a border region for the second foreground portion in another 
image in the time sequence based on the initial border region for the second foreground portion; 
and 

use the initial and automatically generated border regions for masking the second 
foreground portion fi-om the background portion in the digital video. 

83. (new) The computer software of claim 51 fiirther comprising instructions to perform 
the following operations: 

receive an input defining an initial intemal border region, the initial internal border region 
being enclosed by the initial border region, in a first digital image; 

automatically generate an intemal border region for another image in the time sequence 
based on the initial intemal border region; and 

use the initial intemal and the automatically generated intemal border regions for 
masking the foreground portion fi-om the background portion in the digital video. 

84. (new) The computer software of claim 83, wherein the initial intemal border region 
and the automatically generated intemal border region comprise at least a part of the foreground 
portion. 

85. (new) The computer software of claim 83, wherein the initial intemal border region 
and the automatically generated intemal border region comprise at least a part of the background 
portion. 

86. (new) The computer software of claim 83, wherein the initial intemal border region 
and the automatically generated intemal border region comprise at least a part of the foregroxmd 
portion and at least a part of the background portion. 
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87. (new) The computer software of claim 51, wherein the foreground portion is an 

object. 

88. (new) The computer software of claim 51 wherein the instructions to automatically 
generate a border region for another digital image comprise instructions to: 

identify an edge path in the initial border region, the edge path indicating edges of the 
foreground portion in the initial digital image; 

identify an edge path in an automatically generated border region, the edge path 
indicating edges of the foreground portion in the another digital image; 

determine an edge path transformation between the initial digital image and the another 
digital image; and 

apply the edge path transformation to the inside path and the outside path in the initial 
digital image in order to generate a border region in the another digital image. 

89. (new) The computer software of claim 88 wherein the instructions to identify the 
edge path in the another automatically generated border region comprise instructions to: 

identify the edge path using a snake algorithm operating on a previously identified edge 

path. 

90. (new) The computer software of claim 88 wherein the instructions to identify an edge 
path in the initial border region comprise instructions to: 

find points in the border region that have image gradient values exceeding a threshold 

value. 

91. (new) The computer software of claim 90 further comprising instructions to calculate 
an image gradient value for each color channel in the initial border region. 

92. (new) The computer software of claim 88 wherein the instructions to identify an edge 
path in the initial border region comprise instructions to: 

copy the inside path and the outside path; 
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move the copies of the inside path and the outside path towards each other and towards 
edges of the foreground portion using a snake algorithm; and 

identify the edge path as the path where copy of the inside path and the copy of the 
outside path converge into one path. 

93. (new) The computer software of claim 88 wherein the instructions to determine an 
edge path transformation comprise instructions to: 

determine a set of local edge path transformations, the set of local edge path 
transformations together forming the edge path transformation. 

94. (new) The computer software of claim 93 wherein the instructions to determine a set 
of local edge path transformations comprise instructions to determine a local edge path 
transformation for each point in the edge path. 

95. (new) The computer software of claim 93 wherein the instructions to determine a set 
of local edge path transformations comprise instructions to determine a translation, a rotation, 
and a scaling for each local edge path transformation in the set of local edge path 
transformations. 

96. (new) The computer software of claim 88 wherein the instructions to apply the edge 
path transformation to the inside and outside path comprise instructions to apply a set of local 
transformations to each point in the inside path and to each point in the outside path, the set of 
local transformations forming the edge path transformation. 

97. (new) The computer software of claim 96 wherein the instructions to apply a set of 
local transformations comprise instructions to apply a translation, a rotation, and a scaling as 
each local transformation of the inside path and the outside path. 



98. (new) Computer software, tangibly embodied in a computer-readable medium or a 
propagated carrier signal for masking an object fi-om a background in a digital video, the digital 
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video comprising a time sequence of digital images, each image being defined by a plurality of 
pixels, the software comprising instructions to perform the following operations: 

receive a first user input defining a first border region, the first border region including at 
least a part of the object and at least a part of the background in a first digital image; 

receive a second user input defining a second border region, the second border region 
including at least a part of the object and at least a part of the background portion in a second 
digital image; 

interpolate an intermediary border region for an image intermediary in time to the first 
and second digital images; 

use the first, second, and intermediary border regions for masking the object fi"om the 
background in the digital video; 

use the first, second, and intermediary border regions for extracting the object fi-om the 
background in the digital video; and 

place the masked and extracted object against a new background in another digital video. 



